EndoSRR: a comprehensive multi-stage approach for endoscopic specular reflection removal.

PURPOSE: Specular reflections in endoscopic images not only disturb visual perception but also hamper computer vision algorithm performance. However, the intricate nature and variability of these reflections, coupled with a lack of relevant datasets, pose ongoing challenges for removal. METHODS: We present EndoSRR, a robust method for eliminating specular reflections in endoscopic images. EndoSRR comprises two stages: reflection detection and reflection region inpainting. In the reflection detection stage, we adapt and fine-tune the segment anything model (SAM) using a weakly labeled dataset, achieving an accurate reflection mask. For reflective region inpainting, we employ LaMa, a fast Fourier convolution-based model trained on a 4.5M-image dataset, enabling effective inpainting of arbitrarily shaped reflection regions. Lastly, we introduce an iterative optimization strategy for dual pre-trained models to refine the results of specular reflection removal, named DPMIO. RESULTS: Utilizing the SCARED-2019 dataset, our approach surpasses state-of-the-art methods in both qualitative and quantitative evaluations. Qualitatively, our method excels in accurately detecting reflective regions, yielding more natural and realistic inpainting results. Quantitatively, our method demonstrates superior performance in both segmentation evaluation metrics (IoU, E-measure, etc.) and image inpainting evaluation metrics (PSNR, SSIM, etc.). CONCLUSION: The experimental results underscore the significance of proficient endoscopic specular reflection removal for enhancing visual perception and downstream tasks. The methodology and results presented in this study are poised to catalyze advancements in specular reflection removal, thereby augmenting the accuracy and safety of minimally invasive surgery.

Parameter-efficient framework for surgical action triplet recognition.

PURPOSE: Surgical action triplet recognition is a clinically significant yet challenging task. It provides surgeons with detailed information about surgical scenarios, thereby facilitating clinical decision-making. However, the high similarity among action triplets presents a formidable obstacle to recognition. To enhance accuracy, prior methods necessitated the utilization of larger models, thereby incurring a considerable computational burden. METHODS: We propose a novel framework known as the Lite and Mega Models (LAM). It comprises a CNN-based fully fine-tuned model (LAM-Lite) and a parameter-efficient fine-tuned model based on the foundation model using Transformer architecture (LAM-Mega). Temporal multi-label data augmentation is introduced for extracting robust class-level features. RESULTS: Our study demonstrates that LAM outperforms prior methods across various parameter scales on the CholecT50 dataset. Using fewer tunable parameters, LAM achieves a mean average precision (mAP) of 42.1%, a 3.6% improvement over the previous state of the art. CONCLUSION: Leveraging effective structural design and robust capabilities of the foundational model, our proposed approach successfully strikes a balance between accuracy and computational efficiency. The source code is accessible at https://github.com/Lycus99/LAM .

SwinD-Net: a lightweight segmentation network for laparoscopic liver segmentation.

The real-time requirement for image segmentation in laparoscopic surgical assistance systems is extremely high. Although traditional deep learning models can ensure high segmentation accuracy, they suffer from a large computational burden. In the practical setting of most hospitals, where powerful computing resources are lacking, these models cannot meet the real-time computational demands. We propose a novel network SwinD-Net based on Skip connections, incorporating Depthwise separable convolutions and Swin Transformer Blocks. To reduce computational overhead, we eliminate the skip connection in the first layer and reduce the number of channels in shallow feature maps. Additionally, we introduce Swin Transformer Blocks, which have a larger computational and parameter footprint, to extract global information and capture high-level semantic features. Through these modifications, our network achieves desirable performance while maintaining a lightweight design. We conduct experiments on the CholecSeg8k dataset to validate the effectiveness of our approach. Compared to other models, our approach achieves high accuracy while significantly reducing computational and parameter overhead. Specifically, our model requires only 98.82 M floating-point operations (FLOPs) and 0.52 M parameters, with an inference time of 47.49 ms per image on a CPU. Compared to the recently proposed lightweight segmentation network UNeXt, our model not only outperforms it in terms of the Dice metric but also has only 1/3 of the parameters and 1/22 of the FLOPs. In addition, our model achieves a 2.4 times faster inference speed than UNeXt, demonstrating comprehensive improvements in both accuracy and speed. Our model effectively reduces parameter count and computational complexity, improving the inference speed while maintaining comparable accuracy. The source code will be available at https://github.com/ouyangshuiming/SwinDNet.

MRI-based radiomics model for preoperative prediction of extramural venous invasion of rectal adenocarcinoma.

BACKGROUND: Extramural venous invasion (EMVI) is an important prognostic factor of rectal adenocarcinoma. However, accurate preoperative assessment of EMVI remains difficult. PURPOSE: To assess EMVI preoperatively through radiomics technology, and use different algorithms combined with clinical factors to establish a variety of models in order to make the most accurate judgments before surgery. MATERIAL AND METHODS: A total of 212 patients with rectal adenocarcinoma between September 2012 and July 2019 were included and distributed to training and validation datasets. Radiomics features were extracted from pretreatment T2-weighted images. Different prediction models (clinical model, logistic regression [LR], random forest [RF], support vector machine [SVM], clinical-LR model, clinical-RF model, and clinical-SVM model) were constructed on the basis of radiomics features and clinical factors, respectively. The area under the curve (AUC) and accuracy were used to assess the predictive efficacy of different models. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were also calculated. RESULTS: The clinical-LR model exhibited the best diagnostic efficiency with an AUC of 0.962 (95% confidence interval [CI] = 0.936-0.988) and 0.865 (95% CI = 0.770-0.959), accuracy of 0.899 and 0.828, sensitivity of 0.867 and 0.818, specificity of 0.913 and 0.833, PPV of 0.813 and 0.720, and NPV of 0.940 and 0.897 for the training and validation datasets, respectively. CONCLUSION: The radiomics-based prediction model is a valuable tool in EMVI detection and can assist decision-making in clinical practice.

An MRI-only Three-dimensional Cephalometry Protocol based on the Integrated and Modular Architecture of the Human Head.

BACKGROUND: Currently, three-dimensional cephalometry measurements are mainly based on cone beam computed tomography (CBCT), which has limitations of ionizing radiation, lack of soft tissue information, and lack of standardization of median sagittal plane establishment. OBJECTIVES: This study investigated magnetic resonance imaging (MRI)-only based 3D cephalometry measurement based on the integrated and modular characteristics of the human head. METHODS: Double U-Net CycleGAN was used for CT image synthesis from MRI. This method enabled the synthesis of a CT-like image from MRI and measurements were made using 3D slicer registration and fusion. RESULTS: A protocol for generating and optimizing MRI-based synthetic CT was described and found to meet the precision requirements of 3D head measurement using MRI midline positioning methods reported in neuroscience to establish the median sagittal plane. An MRI-only reference frame and coordinate system were established enabling an MRI-only cephalometric analysis protocol that combined the dual advantages of soft and hard tissue display. The protocol was devised using data from a single volunteer and validation data from a larger sample remains to be collected. CONCLUSION: The reported method provided a new protocol for MRI-only cephalometric analysis of craniofacial growth and development, malformation occurrence, treatment planning, and outcomes.

SDA-CLIP: surgical visual domain adaptation using video and text labels.

Background: Surgical action recognition is an essential technology in context-aware-based autonomous surgery, whereas the accuracy is limited by clinical dataset scale. Leveraging surgical videos from virtual reality (VR) simulations to research algorithms for the clinical domain application, also known as domain adaptation, can effectively reduce the cost of data acquisition and annotation, and protect patient privacy. Methods: We introduced a surgical domain adaptation method based on the contrastive language-image pretraining model (SDA-CLIP) to recognize cross-domain surgical action. Specifically, we utilized the Vision Transformer (ViT) and Transformer to extract video and text embeddings, respectively. Text embedding was developed as a bridge between VR and clinical domains. Inter- and intra-modality loss functions were employed to enhance the consistency of embeddings of the same class. Further, we evaluated our method on the MICCAI 2020 EndoVis Challenge SurgVisDom dataset. Results: Our SDA-CLIP achieved a weighted F1-score of 65.9% (+18.9%) on the hard domain adaptation task (trained only with VR data) and 84.4% (+4.4%) on the soft domain adaptation task (trained with VR and clinical-like data), which outperformed the first place team of the challenge by a significant margin. Conclusions: The proposed SDA-CLIP model can effectively extract video scene information and textual semantic information, which greatly improves the performance of cross-domain surgical action recognition. The code is available at https://github.com/Lycus99/SDA-CLIP.

Anatomical network modules of the human central nervous-craniofacial skeleton system.

Anatomical network analysis (AnNA) is a systems biological framework based on network theory that enables anatomical structural analysis by incorporating modularity to model structural complexity. The human brain and facial structures exhibit close structural and functional relationships, suggestive of a co-evolved anatomical network. The present study aimed to analyze the human head as a modular entity that comprises the central nervous system, including the brain, spinal cord, and craniofacial skeleton. An AnNA model was built using 39 anatomical nodes from the brain, spinal cord, and craniofacial skeleton. The linkages were identified using peripheral nerve supply and direct contact between structures. The Spinglass algorithm in the igraph software was applied to construct a network and identify the modules of the central nervous system-craniofacial skeleton anatomical network. Two modules were identified. These comprised an anterior module, which included the forebrain, anterior cranial base, and upper-middle face, and a posterior module, which included the midbrain, hindbrain, mandible, and posterior cranium. These findings may reflect the genetic and signaling networks that drive the mosaic central nervous system and craniofacial development and offer important systems biology perspectives for developmental disorders of craniofacial structures.

Efficacy and Interpretability Analysis of Noninvasive Imaging Based on Computed Tomography in Patients with Hepatocellular Carcinoma After Initial Transarterial Chemoembolization.

RATIONALE AND OBJECTIVES: The objective of this study is to accurately and timely assess the efficacy of patients with hepatocellular carcinoma (HCC) after the initial transarterial chemoembolization (TACE). MATERIALS AND METHODS: This retrospective study consisted of 279 patients with HCC in Center 1, who were split into training and validation cohorts in the ratio of 4:1, and 72 patients in Center 2 as an external testing cohort. Radiomics signatures both in the arterial phase and venous phase of contrast-enhanced computed tomography images were selected by univariate analysis, correlation analysis, and least absolute shrinkage and selection operator regression to build the predicting models. The clinical model and combined model were constructed by independent risk factors after univariate and multivariate logistic regression analysis. The biological interpretability of radiomics signatures correlating transcriptome sequencing data was explored using publicly available data sets. RESULTS: A total of 31 radiomics signatures in the arterial phase and 13 radiomics signatures in the venous phase were selected to construct Radscore_arterial and Radscore_venous, respectively, which were independent risk factors. After constructing the combined model, the area under the receiver operating characteristic curve in three cohorts was 0.865, 0.800, and 0.745, respectively. Through correlation analysis, 11 radiomics signatures in the arterial phase and 4 radiomics signatures in the venous phase were associated with 8 and 5 gene modules, respectively (All P < .05), which enriched some pathways closely related to tumor development and proliferation. CONCLUSION: Noninvasive imaging has considerable value in predicting the efficacy of patients with HCC after initial TACE. The biological interpretability of the radiological signatures can be mapped at the micro level.

MT-FiST: A Multi-Task Fine-Grained Spatial-Temporal Framework for Surgical Action Triplet Recognition.

Surgical action triplet recognition plays a significant role in helping surgeons facilitate scene analysis and decision-making in computer-assisted surgeries. Compared to traditional context-aware tasks such as phase recognition, surgical action triplets, comprising the instrument, verb, and target, can offer more comprehensive and detailed information. However, current triplet recognition methods fall short in distinguishing the fine-grained subclasses and disregard temporal correlation in action triplets. In this article, we propose a multi-task fine-grained spatial-temporal framework for surgical action triplet recognition named MT-FiST. The proposed method utilizes a multi-label mutual channel loss, which consists of diversity and discriminative components. This loss function decouples global task features into class-aligned features, enabling the learning of more local details from the surgical scene. The proposed framework utilizes partial shared-parameters LSTM units to capture temporal correlations between adjacent frames. We conducted experiments on the CholecT50 dataset proposed in the MICCAI 2021 Surgical Action Triplet Recognition Challenge. Our framework is evaluated on the private test set of the challenge to ensure fair comparisons. Our model apparently outperformed state-of-the-art models in instrument, verb, target, and action triplet recognition tasks, with mAPs of 82.1% (+4.6%), 51.5% (+4.0%), 45.50% (+7.8%), and 35.8% (+3.1%), respectively. The proposed MT-FiST boosts the recognition of surgical action triplets in a context-aware surgical assistant system, further solving multi-task recognition by effective temporal aggregation and fine-grained features.

Augmented reality-assisted navigation system contributes to better intraoperative and short-time outcomes of laparoscopic pancreaticoduodenectomy: a retrospective cohort study.

BACKGROUND: Augmented reality (AR)-assisted navigation system are currently good techniques for hepatectomy; however, its application and efficacy for laparoscopic pancreatoduodenectomy have not been reported. This study sought to focus on and evaluate the advantages of laparoscopic pancreatoduodenectomy guided by the AR-assisted navigation system in intraoperative and short-time outcomes. METHODS: Eighty-two patients who underwent laparoscopic pancreatoduodenectomy from January 2018 to May 2022 were enrolled and divided into the AR and non-AR groups. Clinical baseline features, operation time, intraoperative blood loss, blood transfusion rate, perioperative complications, and mortality were analyzed. RESULTS: AR-guided laparoscopic pancreaticoduodenectomy was performed in the AR group ( n =41), whereas laparoscopic pancreatoduodenectomy was carried out routinely in the non-AR group ( n =41). There was no significant difference in baseline data between the two groups ( P >0.05); Although the operation time of the AR group was longer than that of the non-AR group (420.15±94.38 vs. 348.98±76.15, P <0.001), the AR group had a less intraoperative blood loss (219.51±167.03 vs. 312.20±195.51, P =0.023), lower blood transfusion rate (24.4 vs. 65.9%, P <0.001), lower occurrence rates of postoperative pancreatic fistula (12.2 vs. 46.3%, P =0.002) and bile leakage (0 vs. 14.6%, P =0.026), and shorter postoperative hospital stay (11.29±2.78 vs. 20.04±11.22, P <0.001) compared with the non-AR group. CONCLUSION: AR-guided laparoscopic pancreatoduodenectomy has significant advantages in identifying important vascular structures, minimizing intraoperative damage, and reducing postoperative complications, suggesting that it is a safe, feasible method with a bright future in the clinical setting.

CholecTriplet2021: A benchmark challenge for surgical action triplet recognition.

Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of combination delivers more comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and the assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms from the competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%. This study also analyzes the significance of the results obtained by the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.

Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark.

PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. METHODS: To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. RESULTS: F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). CONCLUSION: Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

UMRFormer-net: a three-dimensional U-shaped pancreas segmentation method based on a double-layer bridged transformer network.

Background: Methods based on the combination of transformer and convolutional neural networks (CNNs) have achieved impressive results in the field of medical image segmentation. However, most of the recently proposed combination segmentation approaches simply treat transformers as auxiliary modules which help to extract long-range information and encode global context into convolutional representations, and there is a lack of investigation on how to optimally combine self-attention with convolution. Methods: We designed a novel transformer block (MRFormer) that combines a multi-head self-attention layer and a residual depthwise convolutional block as the basic unit to deeply integrate both long-range and local spatial information. The MRFormer block was embedded between the encoder and decoder in U-Net at the last two layers. This framework (UMRFormer-Net) was applied to the segmentation of three-dimensional (3D) pancreas, and its ability to effectively capture the characteristic contextual information of the pancreas and surrounding tissues was investigated. Results: Experimental results show that the proposed UMRFormer-Net achieved accuracy in pancreas segmentation that was comparable or superior to that of existing state-of-the-art 3D methods in both the Clinical Proteomic Tumor Analysis Consortium Pancreatic Ductal Adenocarcinoma (CPTAC-PDA) dataset and the public Medical Segmentation Decathlon dataset (self-division). UMRFormer-Net statistically significantly outperformed existing transformer-related methods and state-of-the-art 3D methods (P<0.05, P<0.01, or P<0.001), with a higher Dice coefficient (85.54% and 77.36%, respectively) or a lower 95% Hausdorff distance (4.05 and 8.34 mm, respectively). Conclusions: UMRFormer-Net can obtain more matched and accurate segmentation boundary and region information in pancreas segmentation, thus improving the accuracy of pancreas segmentation. The code is available at https://github.com/supersunshinefk/UMRFormer-Net.

A robust and automatic CT-3D ultrasound registration method based on segmentation, context, and edge hybrid metric.

BACKGROUND: The fusion of computed tomography (CT) and ultrasound (US) image can enhance lesion detection ability and improve the success rate of liver interventional radiology. The image-based fusion methods encounter the challenge of registration initialization due to the random scanning pose and limited field of view of US. Existing automatic methods those used vessel geometric information and intensity-based metric are sensitive to parameters and have low success rate. The learning-based methods require a large number of registered datasets for training. PURPOSE: The aim of this study is to provide a fully automatic and robust US-3D CT registration method without registered training data and user-specified parameters assisted by the revolutionary deep learning-based segmentation, which can further be used for preparing training samples for the study of learning-based methods. METHODS: We propose a fully automatic CT-3D US registration method by two improved registration metrics. We propose to use 3D U-Net-based multi-organ segmentation of US and CT to assist the conventional registration. The rigid transform is searched in the space of any paired vessel bifurcation planes where the best transform is decided by a segmentation overlap metric, which is more related to the segmentation precision than Dice coefficient. In nonrigid registration phase, we propose a hybrid context and edge based image similarity metric with a simple mask that can remove most noisy US voxels to guide the B-spline transform registration. We evaluate our method on 42 paired CT-3D US datasets scanned with two different US devices from two hospitals. We compared our methods with other exsiting methods with both quantitative measures of target registration error (TRE) and the Jacobian determinent with paired t-test and qualitative registration imaging results. RESULTS: The results show that our method achieves fully automatic rigid registration TRE of 4.895 mm, deformable registration TRE of 2.995 mm in average, which outperforms state-of-the-art automatic linear methods and nonlinear registration metrics with paired t-test's p value less than 0.05. The proposed overlap metric achieves better results than self similarity description (SSD), edge matching (EM), and block matching (BM) with p values of 1.624E-10, 4.235E-9, and 0.002, respectively. The proposed hybrid edge and context-based metric outperforms context-only, edge-only, and intensity statistics-only-based metrics with p values of 0.023, 3.81E-5, and 1.38E-15, respectively. The 3D US segmentation has achieved mean Dice similarity coefficient (DSC) of 0.799, 0.724, 0.788, and precision of 0.871, 0.769, 0.862 for gallbladder, vessel, and branch vessel, respectively. CONCLUSIONS: The deep learning-based US segmentation can achieve satisfied result to assist robust conventional rigid registration. The Dice similarity coefficient-based metrics, hybrid context, and edge image similarity metric contribute to robust and accurate registration.

Intraoperative laparoscopic liver surface registration with preoperative CT using mixing features and overlapping region masks.

PURPOSE: Laparoscopic liver resection is a minimal invasive surgery. Augmented reality can map preoperative anatomy information extracted from computed tomography to the intraoperative liver surface reconstructed from stereo 3D laparoscopy. However, liver surface registration is particularly challenging as the intraoperative surface is only partially visible and suffers from large liver deformations due to pneumoperitoneum. This study proposes a deep learning-based robust point cloud registration network. METHODS: This study proposed a low overlap liver surface registration algorithm combining local mixed features and global features of point clouds. A learned overlap mask is used to filter the non-overlapping region of the point cloud, and a network is used to predict the overlapping region threshold to regulate the training process. RESULTS: We validated the algorithm on the DePoLL (the Deformable Porcine Laparoscopic Liver) dataset. Compared with the baseline method and other state-of-the-art registration methods, our method achieves minimum target registration error (TRE) of 19.9 ± 2.7 mm. CONCLUSION: The proposed point cloud registration method uses the learned overlapping mask to filter the non-overlapping areas in the point cloud, then the extracted overlapping area point cloud is registered according to the mixed features and global features, and this method is robust and efficient in low-overlap liver surface registration.

Clinical-radiomics predictors to identify the suitability of transarterial chemoembolization treatment in intermediate-stage hepatocellular carcinoma: A multicenter study.

BACKGROUND: Although transarterial chemoembolization (TACE) is the first-line therapy for intermediate-stage hepatocellular carcinoma (HCC), it is not suitable for all patients. This study aimed to determine how to select patients who are not suitable for TACE as the first treatment choice. METHODS: A total of 243 intermediate-stage HCC patients treated with TACE at three centers were retrospectively enrolled, of which 171 were used for model training and 72 for testing. Radiomics features were screened using the Spearman correlation analysis and the least absolute shrinkage and selection operator (LASSO) algorithm. Subsequently, a radiomics model was established using extreme gradient boosting (XGBoost) with 5-fold cross-validation. The Shapley additive explanations (SHAP) method was used to visualize the radiomics model. A clinical model was constructed using univariate and multivariate logistic regression. The combined model comprising the radiomics signature and clinical factors was then established. This model's performance was evaluated by discrimination, calibration, and clinical application. Generalization ability was evaluated by the testing cohort. Finally, the model was used to analyze overall and progression-free survival of different groups. RESULTS: A third of the patients (81/243) were unsuitable for TACE treatment. The combined model had a high degree of accuracy as it identified TACE-unsuitable cases, at a sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) of 0.759, 0.885, 0.906 [95% confidence interval (CI): 0.859-0.953] in the training cohort and 0.826, 0.776, and 0.894 (95% CI: 0.815-0.972) in the testing cohort, respectively. CONCLUSIONS: The high degree of accuracy of our clinical-radiomics model makes it clinically useful in identifying intermediate-stage HCC patients who are unsuitable for TACE treatment.

Double U-Net CycleGAN for 3D MR to CT image synthesis.

PURPOSE: CycleGAN and its variants are widely used in medical image synthesis, which can use unpaired data for medical image synthesis. The most commonly used method is to use a Generative Adversarial Network (GAN) model to process 2D slices and thereafter concatenate all of these slices to 3D medical images. Nevertheless, these methods always bring about spatial inconsistencies in contiguous slices. We offer a new model based on the CycleGAN to work out this problem, which can achieve high-quality conversion from magnetic resonance (MR) to computed tomography (CT) images. METHODS: To achieve spatial consistencies of 3D medical images and avoid the memory-heavy 3D convolutions, we reorganized the adjacent 3 slices into a 2.5D slice as the input image. Further, we propose a U-Net discriminator network to improve accuracy, which can perceive input objects locally and globally. Then, the model uses Content-Aware ReAssembly of Features (CARAFE) upsampling, which has a large field of view and content awareness takes the place of using a settled kernel for all samples. RESULTS: The mean absolute error (MAE), peak-signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM) for double U-Net CycleGAN generated 3D image synthesis are 74.56±10.02, 27.12±0.71 and 0.84±0.03, respectively. Our method achieves preferable results than state-of-the-art methods. CONCLUSION: The experiment results indicate our method can realize the conversion of MR to CT images using ill-sorted pair data, and achieves preferable results than state-of-the-art methods. Compared with 3D CycleGAN, it can synthesize better 3D CT images with less computation and memory.

Preliminary Exploration on the Efficacy of Augmented Reality-Guided Hepatectomy for Hepatolithiasis.

BACKGROUND: The augmented reality-assisted navigation system (AR-ANS) has been initially applied to the management of hepatolithiasis. The current study examines the safety and efficacy of the AR-ANS for hepatectomy in the treatment of hepatolithiasis. It is the first study to assess the preoperative and long-term outcomes of hepatectomy guided by the AR-ANS for hepatolithiasis. STUDY DESIGN: From January 2018 to December 2021, 77 patients with hepatolithiasis who underwent hepatectomy at Zhujiang Hospital of Southern Medical University were included. The subjects were divided into the AR group (n = 31) and the non-AR group (n = 46) according to whether the surgery was guided by the AR-ANS. Clinical baseline features, operation time, intraoperative blood loss, immediate postoperative residual stone rate, postoperative stone recurrence rate at 6 months, and postoperative complications were analyzed. RESULTS: There was no significant difference between preoperative baseline data from the 2 groups (p > 0.05). The AR group had a longer surgical time than the non-AR group (p < 0.001). The intraoperative blood loss in the AR group was lower than in the non-AR group (p < 0.001). Alanine transaminase, aminotransferase, immediate residual stone rate, and stone recurrence rate in the AR group were lower than in the non-AR group (p < 0.05). There was no significant difference in postoperative complications between the 2 groups. CONCLUSIONS: The application of the AR-ANS in hepatectomy for hepatolithiasis has not only achieved satisfactory therapeutic efficacy, but has also shown significant advantages in reducing intraoperative blood loss, immediate stone residual rate, and stone recurrence rate, which has clinical promotion value.

The Medical Segmentation Decathlon.

International challenges have become the de facto standard for comparative assessment of image analysis algorithms. Although segmentation is the most widely investigated medical image processing task, the various challenges have been organized to focus only on specific clinical tasks. We organized the Medical Segmentation Decathlon (MSD)-a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities to investigate the hypothesis that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. MSD results confirmed this hypothesis, moreover, MSD winner continued generalizing well to a wide range of other clinical problems for the next two years. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to scientists that are not versed in AI model training.

Unsupervised learning of depth estimation from imperfect rectified stereo laparoscopic images.

BACKGROUND: Learning-based methods have achieved remarkable performances on depth estimation. However, the premise of most self-learning and unsupervised learning methods is built on rigorous, geometrically-aligned stereo rectification. The performances of these methods degrade when the rectification is not accurate. Therefore, we explore an approach for unsupervised depth estimation from stereo images that can handle imperfect camera parameters. METHODS: We propose an unsupervised deep convolutional network that takes rectified stereo image pairs as input and outputs corresponding dense disparity maps. First, a new vertical correction module is designed for predicting a correction map to compensate for the imperfect geometry alignment. Second, the left and right images, which are reconstructed based on the input image pair and corresponding disparities as well as the vertical correction maps, are regarded as the outputs of the generative term of the generative adversarial network (GAN). Then, the discriminator term of the GAN is used to distinguish the reconstructed images from the original inputs to force the generator to output increasingly realistic images. In addition, a residual mask is introduced to exclude pixels that conflict with the appearance of the original image in the loss calculation. RESULTS: The proposed model is validated on the publicly available Stereo Correspondence and Reconstruction of Endoscopic Data (SCARED) dataset and the average MAE is 3.054 mm. CONCLUSION: Our model can effectively handle imperfect rectified stereo images for depth estimation.